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The  Problem 


The  environment  for  the  military  in  general  is  becoming  more  complex 
and  faster  moving.  There  is  a  need  for  more  military  organizations  which 
themselves  are  more  complex,  faster  acting  and  reacting,  which  design  them¬ 
selves  for  a  mission  and  environment  which  is  unique  today  and  then  redesign 
themselves  for  a  changing  situation  in  the  future.  These  organizations  gen¬ 
erally  utilize  high-technology  and  their  membership  is  educated.  What  methods 
can  these  units  use  to  sense  and  adapt  rapidly  and  properly  to  a  changing 
environment?  Which  combinations  of  technology,  work  structures,  organization 
designs  and  procedures  are  appropriate  in  which  environment?  In  studying 
this  problem,  how  can  the  field  researcher  systematically  study  both  the 
development  of  techniques  and  procedures  to  help  these  organizations  self- 
adapt  and  also  study  the  impact  these  new  techniques  have  on  organizational 
performance? 

To  put  it  differently,  what  measurement  methods  are  appropriate  in  a 
formative  and  summative  evaluation  study  in  which  the  sample  size  is  one, 
environmental  variables  affecting  the  sample  are  changing,  there  are  no 
comparable  sample  organizations,  and  the  treatment  itself  is  designed  to 
Impact  on  all  aspects  of  the  sample  and  its  relationships  with  its  environ¬ 
ment? 


If  the  organization  is  more  effective,  whatever  that  is,  after  the 
treatment,  should  the  treatment  be  considered  as  possibly  having  had  an 
effect?  More  importantly,  how  does  one  know  if  one  properly  and  competently 
administered  the  treatment? 

Illustration  of  the  Problem 


To  illustrate  the  problem  and  provide  us  with  material  for  discussion, 

I  shall  relate  a  recent  Army  Research  Institute  project  in  sociotechnical 
systems  analysis  as  applied  to  the  World-wide  Military  Command  and  Control 
System,  Data  Processing  Center  -  Europe  (WWMCCS,  DPC-E) . 

The  military  has,  like  the  rest  of  western  society,  an  increasing 
number  of  specialized,  high-technology  organizations  which  must  adapt  to 
changing  technology,  staffing,  and  mission  environments.  Examples  are  the 
data  processing  centers,  signal  units,  and  the  air  defense  units  in  West 
Germany.  In  1978,  sociotechnical  systems  analysis  methods  seemed  appropriate 
as  a  way  of  helping  these  units  assess  and  redesign  themselves,  but  practical 
research  remained.  A  contract  was  let  to  Dr.  Bill  Pasmore  at  Case  Western 
Reserve  University’s  School  of  Management.  Pasmore  was  to  develop  and  try 

^The  views  expressed  in  this  paper  are  those  of  the  author  and  do  not 
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out  a  model  of  soclotechnical  systems  analysis  adapted  to  the  military.  The 
target  unit  would  be  the  WWMCCS,  DPC-E.  I  was  the  Contracting  Officer's 
Technical  Representative  and  was  on  site  the  first  year  of  the  project  as 
a  consultant.  Note  that  the  purpose  was  to  develop  and  try  out  a  model 
adapted  to  the  military,  not  demonstrate  the  effectiveness  of  an  already 
developed  model. 

Soclotechnical  systems  theorists  argue  that  the  human  and  technological 
subsystems  of  organizations  operate  according  to  different  sets  of  laws, 
and  that  making  one  as  efficient  as  possible  may  have  unfavorable  effects  on 
the  operation  of  the  other;  therefore,  for  the  organization  to  operate  smoothly 
overall,  it  is  necessary  to  design  each  system  with  respect  to  the  other.  To 
accomplish  this  joint  optimization,  the  Commander,  with  the  help  of  recommen¬ 
dations  from  an  expert  consultant  and  unit  members,  over  time,  reassesses  and 
modifies  where  necessary  the  unit's  organization  structure  and  policies,  tech¬ 
nologies,  task  structures,  and  work  methods. 

The  WWMCCS,  DPC-E  is  located  in  Heidelberg,  West  Germany  in  the  Headquarters 
of  the  US  Army  -  Europe.  It  serves  as  a  communication  link  for  the  Joint  Chiefs 
of  Staff  and  as  a  command  and  control  information  systems  developer  for  the 
headquarters.  The  director  of  the  facility  saw  the  following  challenges  in 
the  years  ahead:  (1)  decreasing  number  and  skill  level  of  personnel;  (2)  need 
for  a  mobile  capability  in  addition  to  the  present  fixed  station  posture; 

(3)  increases  in  total  demand  for  services  and  a  changing  type  of  demand  from 
users. 

The  overall  project  strategy  could  be  summarized  as  a  recurring  cycle  of 
analysis,  recommendations,  and  implementation.  Organization  members  at  all 
levels  would  be  heavily  involved,  with  the  consultant  passing  on  necessary 
skills  to  members  so  they  could  continue  the  process  after  the  consultant  was 
gone. 

First  Data  Collection 

At  the  beginning  and  end  of  the  20-month  time  frame  during  which  the  con¬ 
sultants  periodically  visited  the  unit  and  carried  out  the  intervention,  an 
intensive  data  gathering  operation  was  conducted  using  a  survey  questionnaire, 
interviews,  observation,  and  archival  data.  The  survey  instrument,  very  broad 
ranging  yet  in-depth,  measured  organizational  climate,  work  and  job  character¬ 
istics,  satisfaction,  formalization  of  procedures,  distribution  of  power,  and 
demographics.  The  initial  data  suggested  the  unit  was  in  fairly  good  shape. 
Employees  were  satisfied  with  the  design  of  their  jobs,  supervision,  co-workers, 
rewards.  They  were  motivated  by  their  work,  were  learning  useful  skills,  and 
liked  their  location. 

Seventy-eight  percent  of  the  unit  members  were  interviewed.  Questions 
were  developed  by  both  the  consultants  aud  two  "core  groups."  The  core  groups 
were  parallel  organizations  within  the  unit  formed  to  provide  an  additional 
sensing/feedback  mechanism  to  the  command  group.  They  were  composed  of  8  to 
12  members,  recruited  from  all  levels  and  parts  of  the  unit.  Interview  data 
showed  individuals  working  in  different  parts  of  the  unit  held  widely  diver¬ 
gent  views  about  it.  While  the  interviews  again  demonstrated  a  general 
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satisfaction  with  the  organization,  they  did  also  show,  in  various  sections 
of  the  unit,  that  there  was  dissatisfaction  with  the  work  itself,  training, 
fairness,  measurement  of  productivity,  cooperation  across  boundaries,  rela¬ 
tions  with  users,  and  instability  of  personnel. 

The  third  major  assessment  was  of  the  technical  systems.  Adapting 
methods  used  primarily  by  industrial  engineers,  the  consultants  assessed 
role  descriptions,  skills  needed,  team  and  individual  goals,  task  activities, 
information  used,  problems  encountered,  and  recommendations  for  changes  in 
task  procedures.  A  variance  matrix  was  developed  which  showed  impact  of 
problems  in  one  part  of  the  organization  on  other  parts.  From  this  analysis, 
needed  changes  in  organization  design,  policies,  and  procedures  could  be 
identified. 

Thus,  while  there  was  a  clear  need  in  the  director's  mind  to  prepare  for 
changes  in  the  future,  the  present  state  of  the  unit  was  reasonably  satisfac¬ 
tory.  It  was  operating  effectively  and  to  the  general  satisfaction  of  most 
of  its  members.  The  need  for  change  was  not  readily  apparent  nor  was  change 
desired  by  many  personnel.  The  critical  issues  were  to  prepare  it  for  the 
future  and  to  help  it  improve  its  productivity  without  lessening  its  quality 
of  work  life.  The  unit  would  be  ideal  for  the  development  of  a  model  for 
soclotechnical  systems  analysis,  but  far  less  desirable  as  a  site  to  demon¬ 
strate  the  power  of  the  model. 

Changes  Made 

Several  major  and  many  minor  changes  were  made  In  the  unit.  Each  change 
was  thoroughly  thought  out  and  talked  over  by  the  core  group,  by  management 
and  the  consultants.  Decisions  were  made  to  move  from  a  fairly  standard 
organization  design  to  a  matrix  design;  this  In  turn  would  allow  the  forma¬ 
tion  of  small  temporary  project  teams  and  more  formal  emphasis  on  skill 
training.  Performance  contracts  for  both  project  and  skill  development  were 
to  be  agreed  upon,  and  rewards  would  be  tied  to  performance.  The  core  group 
was  to  be  continued. 

Interview  data  were  gathered  by  the  core  group  m.-imbers  and  the  consul¬ 
tants  throughout  the  time  period  of  the  change,  with  a  formal  interviewing  of 
about  half  the  unit  one  year  after  the  start  of  the  project.  It  was  obvious 
that  the  change  process  here  was  a  slow  and  difficult  one,  and  the  unit  was 
still  learning  and  deciding  about  itself. 

Later  Data  Collections 


About  five  months  later,  the  WWMCCS  DPC-E  took  part  in  a  major  exercise. 
This  is  viewed  as  the  real  test  of  the  system's  capability.  Compared  to 
years  past,  user  demand  exploded.  New  technology  put  the  unit's  services 
in  much  greater  demand  for  programming,  training  users,  and  installing  tele¬ 
communications  equipment.  At  the  same  time,  there  was  a  slight  decrease  in 
unit  personnel  strength  and  skill  level.  The  unit  trained  and  serviced  sig¬ 
nificantly  more  users  and  increased  programmer  support  of  exercise  activi¬ 
ties  by  15  percent  while  maintaining  existing  duties. 
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Four  months  later,  a  second  survey  was  readrainistered  to  the  unit.  Note 
there  was  about  80  percent  turnover  in  respondents  between  the  first  and  sec¬ 
ond  survey.  There  were  no  signiifcant  changes  in  organization  climate,  sat¬ 
isfaction,  or  other  attitudes.  The  unit  will  be  looked  at  again  in  the  Spring 
of  1982  in  much  the  same  way. 


My  colleague,  U1  James,  and  an  associate,  looked  at  the  unit  intensively 
for  about  2h  days  in  August  1981.  Using  structured  interview  methodology, 
they  found  little  awareness  of  change  or  improvement  among  unit  members. 

Unit  members  attributed  the  hard  outcomes  of  improved  exercise  performance 
not  to  the  sociotechnical  systems  operation  but  to  the  good  management  and 
hard  effort  of  the  unit.  There  was  some  question  as  to  whether  the  change 
to  the  matrix  was  either  very  smooth  or  didn't  occur. 


Thus  we  have  a  difference  of  opinion  on  what  occurred  and  why  it  occurred. 
Different  methods  of  data  gathering  at  different  points  in  time  revealed  dif¬ 
ferent  aspects  of  the  organization.  To  what  extent  are  these  different  view¬ 
points  correct?  How  does  one  measure  an  organization  as  this  one  is? 

( 

Restatement  of  the  Problem 


To  sum  up,  field  research  in  organizations  is  costly  in  time,  money,  and 
the  potential  for  error.  Often,  it  is  desirable  to  get  a  formative  and  summa- 
tive  evaluation  of  a  new  treatment  at  the  same  time.  Yet  in  this  problem  the 
unit  is  chosen  because  it  is  changing  in  its  technology,  personnel,  and  envi¬ 
ronment,  and  is  unique.  What  generalizations  can  be  made  from  such  a  study? 
The  problem  area  seems  a  proper  field  of  inquiry,  yet  there  are  no  accepted 
or  proper  methods  of  inquiry.  What  measurement  methods  are  appropriate  to 
XQ  determine  how  well  the  treatment  is  being  administered  and-42).  determine 
impact  of  the  treatment,  when  the  sample  organization  is  chosen  because  it 
is  unique,  constantly  changing,  and  important. 

Major  Reference 


Pasmore,  W. ;  Shani,  R. ;  and  Kaplan,  M.  Sociotechnical  Approaches  to 
Organization  Change  in  USAREUR.  US  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences,  Draft  Final  Report,  1981. 
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Alternative  Approaches  to  the  Design  and  Analysis  of  Objectlves- 
Referenced  Competency  Tests 


O'Neil,  Harold  F.,  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences,  Alexandria,  Virginia  (Chair);  Baker,  Eva  L.,  Choppin, 
Bruce,  &  Quellmalz,  Edys  S.,  UCLA  Graduate  School  of  Education,  Los 
Angeles,  California.  (Thurs.  P.M.) 


The  need  to  describe  trainee  competency  in  relation  to  specific 
tasks  and  skill  levels  requires  approaches  to  the  construction  and 
analysis  of  criterion  test  situations  quite  different  from  current 
methodologies.  In  "Matching  Tests  with  Instruction,"  Eva  L.  Baker 
will  describe  the  design  of  a  system  for  matching  explicit  content 
requirements  of  Instructional  and  testing  tasks,  estimating  their  "fit” 
and  assessing  the  cost  of  the  process.  Edys  S.  Quellmalz  will  draw 
upon  cognitive  learning  research  to  describe  a  continuum  along  which 
features  of  test-like  tasks  can  approximate  the  information  bases  and 
cognitive  operations  trainees  must  activate  in  actual  situations. 
Alternative  validation  procedures  will  also  be  described.  Bruce 
Choppin  describes  how  latent  trait  modeling  of  test  responses  can  be 
used  to  define  a  scale  of  performance  in  terms  of  specific  tasks  and  to 
locate  an  individual  examinee  within  a  region  of  the  scale. 
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